GELFAND TRANSFORMS 
OF S0(3)-INVARIANT SCHWARTZ FUNCTIONS 
ON THE FREE NILPOTENT GROUP 

VERONIQUE FISCHER AND FULVIO RICCI 



Abstract. The spectrum of a Gelfand pair {K k N, K), where is a nilpotent group, can 
be embedded in a Euchdean space. We prove that in general, the Schwartz functions on 
the spectrum are the Gelfand transforms of Schwartz AT-invariant functions on A^. We also 
show the converse in the case of the Gelfand pair (5*0(3) k A'3.2, 50(3)), where A'3.2 is the 
free two-step nilpotent Lie group with three generators. This extends recent results for the 
Heisenberg group. 

1. Introduction 

Let be a connected, simply-connected, two-step nilpotent Lie group. Let K he a. 
compact group acting by automorphism on A^. We assume that {K x A^, K) is a Gelfand 
pair. The Gelfand spectrum can be homeomorphically embedded in a Euclidean space as 
follows. 

Let D(A^)'^ be the algebra of left-invariant and i^-invariant differential operators on A^ 
and {-Di, . . . , Dg} a finite set of essentially self-adjoint generators of D(A^)^. We call V the 
ordered family {Di, . . . , Dg). To each bounded i^-spherical function on A^ we assign the 
g-tuples of eigenvalues = . . . , i.e. such that DjCp = ^j{(f))(f). The set 

of such g-tuples is in 1-1 correspondence with the Gelfand spectrum and the topology 
induced on it from M'' coincides with the Gelfand topology |Fer07j . 

We define the Gelfand transform Q : L^{N)^ Co(Sx)) by: 

Jn 

We are interested in the following conjecture: 
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Q establishes an isomorphism between S{N)^ and S(Jlx>) 
(as Frechet spaces) 

The validity of this statement is independent of the choice of V (see Section [s]); therefore 
once proved for one particular choice of P, it is true for any choice of T>. 



Proposition 3.3 of this paper shows the continuous inclusion S(T,x>) ^ G{<S{N)^). This 
property is already known for the case of the Heisenberg group |ADR08[ Theorem 5.5]. 
The proof relies on a generalisation |ADR08l Theorem 5.2] of Hulanicki's Schwartz kernel 
Theorem |Hul84] . 

The converse inclusion has been recently shown for any Heisenberg Gelfand pair |ADR08] 
and we prove it here for (5*0(3) x Ai's 2, 5*0(3)) where A^3^2 is the free two-step nilpotent Lie 
group with three generators; we realise A^^s 2 as Mi^xM^ 

{0} X M.y being the centre. It is known 
that (S0(3) X N3^2,SO{3)) is a Gelfand pair |BJR90l Theorem 5.12]. We will give explicit 
formulae for a family of three essentially self-adjoint operators V that generate D(A"3^2) , 
the bounded spherical functions and their corresponding eigenvalues for T>. 

Our goal here is to prove that for any Schwartz 5*0 (3) -invariant function F G S{N-3^2)^^^^\ 
there exists a Schwartz extension of its Gelfand transform: 

i.e. 3feS(R') f\^, = gF 

In the proof, we will use the known result for the three-dimensional Heisenberg group Hi. 
For this, let us consider A^', the quotient group of A'3^2 by the central subgroup M^^^^ and 
K', the stabiliser of M^^^ in 5*0(3). We will see that A^' is isomorphic to Hi x ]R^.3, and K' 
is isomorphic to Ui x Z2 (see Section [2]). The Gelfand transform for the pair {K' x N',K') 
will be denoted by Q'. 

Whenever it makes sense, we denote by TZF the function on A^' given by integration 
of a function F of A'3^2 on the central subgroup M.'^y_^y^y The operator 71 maps 50(3)- 
invariant functions on A^3^2 to K'-invariant functions on A^', and Schwartz functions on A'3^2 
to Schwartz functions on A^'. TZ is 1-1, but does not send 5(A"3^2)'^'^''^^ onto S{N')^ (see 



Proposition 4.3). The definition of TZ can be extended to left-invariant differential operators 



in such a way that TZ{DF) = {TZD){TZF) for any left-invariant differential operators D 
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and any smooth compactly supported functions F on A^. We will see that the image of 
the operators in D by 7?. completed with —d'^^ gives a family V of essentially self-adjoint 
generators for D(A^')^ . Again we will give explicit formulae for T>' , the bounded spherical 
functions, and their corresponding eigenvalues for V . 

The spectrum of {K' x A^', K') can be projected onto the spectrum of {S0{?>) x A'3^2, 5*0(3)) 
in the following sense: composing an homomorphism of L^{N')^ with TZ provides a mapping 
n : Sx)' Sd between the two spectra, that is completely explicit here. In fact 11 maps 
continuously Sx?' onto S^), but is 1-1 only on the regular part of the spectrum (see Section [2]). 

For any Schwartz S'0(3)-invariant function F G 5(A"3 2)'^'^''^'', we have: 

g'{nF) = gFou. 

The existence of a Schwartz extension to for Q'{TZF), can be deduced easily from the 
Heisenberg case |ADR07| lADROSj : it does not imply directly the existence of a Schwartz 
extension for QF but is constantly used all along the proof. 

This article is organised as follows. In Section [2| we introduce the notations and the 
basic facts concerning the Gelfand spectra of {S0{3) x A'3 2,5'0(3)) and {K' x N',K'). In 
Section [3} we give some general settings and the precise statements of our results. In Section 
4 we describe TZ and the restriction mappings. In Section 5 we give the proof of Theorem 5 
using an extension of a mean value formula due to Geller in the case of the Heisenberg group 
|Gel80] ■ In the appendix, we give, for completeness, detailed proof of some results appearing 
in this paper and concerning differential operators and functional calculus on them. 

2. The Gelfand spectra of (^0(3) x N3^2,SO{3)) and {K' x N\K') 
We realise A'3^2 as x endowed with the law: 

{x,y).{x',y') = {x + x',y + y' + ^x Ax'), 

where A indicates the usual wedge product in M'^. A/'3_2 denotes its Lie algebra. For j = 1, 2, 3, 
let Xj be the left-invariant vector field on A^ that equals d^^ at 0, and Yj the left-invariant 
vector field on A^ that equals dy^. {Xj)^^^ ^ 3 and {Yj)^^-^ ^ 3 form the canonical basis of A/3,25 
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and satisfy: 

[Xi,X2] = y3 , [X3,Xi] = F2 , [X2,X3] = yi. 

The group 5*0(3) acts on and thus on A^3^2 by acting simuhanously on each copy of 
M?. One checks easily that this action is by automorphisms on Ai'3 2. {S0{3) x Ai's 2, 5*0(3)) 
is a Gelfand pair |BJR90[ Theorem 5.12]. 

Let us define the sub-Laplacian L, the central Laplacian A and a third operator D by: 

j=l j=l j=l 

In Section |3| we will show that these operators form a family T) = {L,A,D) of essentially 
self-adjoint operators that generate D(A^3 2)'^'^''^''. 

The bounded spherical functions and their corresponding eigenvalues for V are known 
explicitly. Let us define some notation first: for any vector x = (xi, 0:2, 0:3) G M^, we will 
write X or x~for (xi,X2) and, occasionally, [xjg for X3. Ci{u) = (l//!)e"/^((i/(i-u)'-u'e~" denotes 
the Z-Laguerre function of order on M. Then the bounded spherical functions on A'"3^2 are: 

^x,iA^,y) = [ e-*^[^-^^l3/:Y^|[A;.x]f )e-"'[^-^l3rffc , A > 0, / G N, r G M. 

Jk&SO{3) ^ 

and 

0o_^(a;, y)= [ e-^^t'^-^la dk = ^-^^^^^dk , R > 0; 

their eigenvalues for V are given by: 

/x^,,^ = (A(2/ + l) + r^A^Ar), 
= (/^^0,0). 

The Gelfand spectrum S^? of (50(3) x Ai'3_2, 5*0(3)) is then realised as the union of the 
collection of the /i,/.;^;^, A > 0, / G N, r G M (the regular part of the spectrum), with the 
collection of the /x,^^, _R G M (the singular part of the spectrum). Calling {rji,rj2,rj^) the 
coordinates corresponding to L, A, D respectively, Sx) is then the union, for Z > 0, of the 
surfaces Vi defined by the equation rjl = rj2{rji — {21 + l)y^). They all meet together on 
the positive ?72-axis, the singular part of S^). 
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A^' is the quotient group of N3^2 by the central subgroup M^^^ ^^y, we reahse A^' as x 
endowed with the law: 



{x,t).{x',t') = {x + x',t + t' + ^[x Ax%). 



Af' denotes its Lie algebra; this is a quotient of A/3,2 by MYi©MF2 and we denote q : A/3,2 A/"' 
the quotient mapping. q{Xj) = Xj is the left-invariant vector field on A^' that equals 
at 0, j = 1,2,3; ^(^1) = ^(^2) = 0, and qiY^,) = T is the left-invariant vector field on 

that equals dt- In particular X'^ = d^g lies in the centre of Af'. {^j)j-i2s form 
the canonical basis of Af'. It is easy to see that A^' is isomorphic to Hi x M. Let K' be the 
stabiliser of %^^y^) C in S0{3). The group/sT' is 5(0(2) x 0(1)) ^ 0(2) = f/i x Z2. 

{K' K A^', K') is a Gelfand pair and its bounded spherical functions are explicitly known: 

(f)'^^^^{x,t) = cos{Xt + rX:r3)jCi{^\[k.x]~\'^)dk , A > 0, / G N, r G M, 

and 

0C,r(a;> y) = JoiC\x\) cos(ra;3) , C,r eR, 

Jo being the Bessel function of order 0. 
We define the following operators: 

3 

L' = ~J2^f ' A' = -^' , D' = -X'^T. 
The operators L', A', D' and —X'^^ are fT'-invariant, essentially self-adjoint and generate 



(A^')^' (see Proposition lJUj and Subsection [5^. We set the family V = {L' , A', D', -X'^ 



f2\ 



^3 



The eigenvalues of the bounded spherical functions for V are given by: 

= (A(2/ + l)+r2,A2,Ar,r2), 
= (C' + r^0,0,r2). 



As in the case of (S0(3) x Ar3,2, 50(3)), the Gelfand spectrum S^, of {K' x A^', K') is then 
realised as the union of a regular and a singular part: the regular part is the collection of 
the ^ ^, A > 0, Z G N, r G M, and the singular part is the collection of the /^</)'^ ^, ^ I^- 
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With coordinates ''72, ''73, ''74) corresponding to L' , /S! , D' , — X'.^ respectively, S^, is the 
union of the set 0, 0, 774) : < < rji\ (the singular set) and the two-dimensional 
surfaces F^, / > 0, defined by the system of equations 

%' = ^2(r/i-(2/ + l)7^) 

Notice that the projection onto the hyperplane 774 = parallel to the ?74-axis maps E^, 
onto Sx), and is bijective between the two regular sets. This fact, alluded to already in the 
introduction, will be relevant in view of the mapping TZ defined in Subsection 4.1. 

Let us give an equivalent and intrinsic point of view of this fact. As explained in the 
introduction, the spectrum of {K' x A^', K') can be projected in the following sense onto the 
spectrum of {S0{3) x A^^s 2, S0{3)): the composition of an homomorphism of L^{N')^ with 



TZ (see also Subsection 4. 1 ) provides a mapping between the two Gelfand spectra. Realising 
the Gelfand spectra as explained in the introduction (see also Section [sj, this mapping 
n : S©/ —>■ Sx) is then given by: 

^(A(2/ + l)+r^A^Ar,r2) = (A(2/ + 1) + r^, A^, Ar) , 
U{C + r\0,0,r') = (C' + r^O,0). 

n maps continuously S^)' onto J^-p. Moreover 11 maps homeomorphically the regular part of 
Sx)' onto the regular part of S^); H maps the irregular part of onto the irregular part of 
Sxj, but this correspondence is not 1-1. 

3. Results 

In this section, we describe the general settings of our work and explain the conjecture 
g{S{Nr) =5(S2,). We will give the precise statement of our main result in Theorem 



3.5 



Let A^ be a connected, simply-connected Lie group. A/" its Lie algebra, exp : JV ^ N 
the exponential mapping and (-Ej)f^^ a basis of A/". The canonical basis (-E'i)f^^ of Af being 
chosen, this induces a Lebesgue measure dX on A/" and, via the exponential map, a Haar 
measure dn on A^; the spaces Lp{N) are defined with respect to this Haar measure. When A^ 
is a graded Lie group, following |FS82[ chl.D], we fix a homogeneous gauge |.| on A^ and we 



|u=0 
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keep the same notation for the basis (Ej) of A/" and the associated left-invariant vector fields 
on N; we set the following family of semi-norms parametrised by a G N on the Schwartz 
space S{N) which induces the usual Frechet space structure on S{N): 

\\F\l^^= sup il + \n\r\E'Fin)\. 

neN,d{I)<a 

V{N') denotes the algebra of polynomials on M with real coefficients where M is then 
identified with the Euclidean vector space W . D(A^) denotes the algebra of real left-invariant 
differential operators on N , as operators defined on C^{N), the space of smooth, compactly- 
supported functions on A^. 

To P eV{Af), we associate Dp eB{N) by: 

p 

DpF{n)= P{i-^du)F{nexp{J2ujEj)) 

i=i 

We obtain the symmetrisation mapping P ^ Dp, that is a linear isomorphism between the 
algebras D(A^) and V{U) [HelSil Ch.II Theorems 4.3 and 4.9] 
In the appendix we show: 

Proposition 3.1. If{Kt<N,K) is a G elf and pair, each operator ofD^N)^ is essentially self- 
adjoint, that is, it admits a unique self-adjoint extension to an unbounded operator of L'^{N). 

Furthermore the operators of I]){N)^ commute strongly, in the sense that the spectral 
resolutions of their self-adjoint extensions commute. 

We will use the same notation for an operator of D(A^)'^ and its self-adjoint extension. 

By Hubert's Basis Theorem, if a group K acts orthogonally on some Euclidean space W, 
the algebra V{W)^ of i^-invariant polynomials on MJ' is finitely generated |Hel84t Ch.II 
Corollary 4.10]. If pi, . . . , pg is a set of generators, we call {pi, . . . , p^} a Hilbert basis 
for {W,K) and p = (pi,...,pg) the corresponding Hilbert mapping. Furthermore, if p = 
(pi, . . . , pq) and p' = {p[, . . . , p'g,) are two Hilbert mappings for {Rp,K), then there exists 
Q = (Qi) • • • ) Qq)^ Qj ^ VCR.^'), such that p = Qop' (and viceversa). We will make extensive 
use of G. Schwarz's Theorem |Sch75] : every K-invariant smooth function on MP can be 
expressed as a smooth function of a Hilbert basis p of {W,K). In other words, the Hilbert 



8 V. FISCHER AND F. RICCI 

map, p, induces an application given by p*{h) = h o p. Moreover p* is a linear continuous 
mapping from C°°(M9) onto C'^iW)^, and also from S{W) onto S{W)^ |ADR08l Theorem 
6.1]. 

Assume that {K x A^, K) is a Gelfand pair. Any family of generators of D(A^)''^ is obtained 
as the symmetrisation of a Hilbert basis, and conversely, if {pi, . . . , pq} denotes a Hilbert 
basis for (A/", K), then {Dp^, . . . , -Dp,} is a set of generators of D(A^)^. 

Let us fix {pi, . . . , Pq) an ordered Hilbert basis for [M^K), to which we associate the 
ordered family of operators Vp = {Dp^, . . . , -Dp,). We denote by Sx^^, the set of the g-tuples 
of eigenvalues /i(0) = . . . of Vp for the bounded K-spherical functions on 

A^. As mentioned in the introduction and proved in |Fer07j . Sx)^ is the realisation of the 
Gelfand spectrum associated to "Dp, in the sense that the set Sx)^ of such g-tuples is in 1-1 
correspondence with the Gelfand spectrum and the topology induced on it from coincides 
with the Gelfand topology. In the appendix, we will show that Sxj^ is also the joint spectrum 
of Vp- 

Proposition 3.2. Let (pi, . . . , pq) he an ordered Hilbert basis for (A/", K). The joint spectrum 
of the family of strongly commuting, self-adjoint operators T>p = {Dp-^, . . . , -Dp,) is ^Vp- 

For a closed subset E of M'^, S{E) denotes the space of restrictions to E of Schwartz 
functions, endowed with the quotient topology of S{W)/{f : f\E = 0}; we will often define 
a class in this quotient as being given as the restriction of a Schwartz function on W^. The 
spectrum Sx) is a closed subset of W^. We are interested in the conjecture S{N) ~ SiT^x,). 
The existence of a polynomial mapping between two Hilbert mappings implies that the 
validity of this conjecture is independent of the choice of V (see |ADR08| Section 3]). The 
continuous inclusion iS(Sx)) ^ Q{S{N)^) relies on the following statement, which is a 
generalisation of Hulanicki's Schwartz Kernel Theorem proved in the appendix: 

Proposition 3.3. Let (pi, . . . , Pq) be an ordered Hilbert basis for (A/", K), andVp = {Dp-^, . . . , Dp^) 
the associated family of operators. 
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Let m be in 5(]R^). The operator m{T>p) is a convolution operator with a K -invariant 
Schwartz kernel M = Mjn,Vp ^ 5(A^)^.' 

WFeL'^iN) m{Vp)F = F*M. 

The Gelfand transform of M is: 

Furthermore the mapping m E S(MS) h-^ Mm,Vp & S{N)^ is continuous. 

For the Gelfand spectra of Heisenberg groups or the free two-step nilpotent Lie groups, 
the inclusion of the spectrum in the image of the Hilbert mapping: 

J^Vp C imp, 

is true, independently of the choice of the Hilbert mapping p (but we do not know if it is 
true in general). Here we will use this property only in the case of N' = Hi x M, where it is 
known, the spectrum and the Hilbert mapping being explicit. 

Lemma 3.4. The polynomials and x ■ y generate the algebra of polynomials on 

M.I X My that are invariant under the simultaneous action of SO {3) on each copy o/M^. 

Proof. If P{x,y) is an S'0(3)-invariant poljTiomial on M^. x M^, then for each independent 
vectors x,y E M^, we have P{x,y) = P{—x,~y) because the linear transformation that 
equals —Id on the vector space spanned by x and y, and 1 on the orthogonal complement 
line, is in 5*0(3); this shows that P is invariant under — IdiRS, and thus also under the 
simultaneous action of 0(3) on each copy of M^. This implies: 

V{Rl X Rlf^^^^ = V{Ri X Rlf^^\ 

By |(;W981 Theorem 4.2.2.(1)], P(M^, x M3)0(3) gp^nned by \y\'^ and x-y. □ 

Thus p{x,y) = (Ixp, ■ y) gives a Hilbert mapping for (A/3,2, 5'0(3)). We compute 

easily that the associated family of operators by symmetrisation is P = (L, A, D) defined 
in Section [2| where we give also an explicit description of the associated realisation of the 
Gelfand spectrum. 
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Theorem 3.5. The Gelfand transform of any Schwartz S0{?))- invariant function on N^^2 
admits a Schwartz extension to M.^: 

Moreover the mapping F G X)) is an isomorphism of Frechet 

spaces. 

The group admits a shghtly bigger group of automorphisms than S0{3), namely 0(3) 
acting by: 

k{x,y) = (^kx, {det k)ky) , k G 0(3), 

It is easily verified that {|a;p, {x ■ y^} gives a Hilbert basis for (A/3,27 0(3)) and the 
associated family of operators by symmetrisation is P = {L, A, D"^). Following the same 
lines as in |ADR08l Section 8], we have the following. 

Corollary 3.6. The Gelfand transform is an isomorphism between S{N3 2)^^^^ CLnd S{T,f,) 
as Frechet spaces. 

From now on, will stand for A"3 2 and K for 50(3). 

4. TZ AND RESTRICTION MAPPINGS 

4.1. The mapping IZ. In the introduction, we denoted by IZF the function on A^' given by 
integration of a function F of A^ on the central subgroup M^^^ whenever it makes sense, 
for example on L^{N). It is sometimes convenient to consider TZ as acting between functions 
defined on the Lie algebras, rather than on the groups. We will do so without any further 
mention. The operator TZ maps A^-invariant functions on A^ to A''-invariant functions on A^', 
integrable functions on A^ to integrable functions on A^' continuously, Schwartz functions 
on A^ to Schwartz functions on A^' continuously. It respects convolution on the groups and 
abelian convolution on the Lie algebras. 

We extend the definition of TZ to left-invariant differential operators on A^. Let D = Dp E 



i{N), P G ■P(A/'). Easy changes of variables, see e.g. (10) below, show: 



VF G C^{N) , VG G C^{N') {TZ{DF), G) = {TZF, Dp^^,G) 
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We are led to define 7^ : D(A^) B{N') by: TZDp = Dq where Q = P\j^> is the restriction 
of P to N' . For any D G ©(iV), TZD is the only differential operator T on N satisfying: 

VF e C^{N) n[DF] = TUF. 

The mapping TZ on functions is dual to the restriction mapping from H to H' in the 
following sense. Let us denote Ty and Tt the Fourier transform with respect to the variables 
1/ G and t G M respectively given by: 

J^tGixJ) = [ G{x,t)e-'*-^dt; 
Jr 

whenever it makes sense for a function G on A^' and a function F on N, identified with 
functions on J\f' and J\f respectively, we have: 

(1) G = nF^J^tG = TyF\j^, 

In the following subsection, we describe the restriction mapping. 

4.2. Restriction and radialisation mappings. For a function / on M, we denote by 
Rest / = f\j^i its restriction to M' . We set: 

AA„ = X (Kj\{o}) and AA^ = x (MAW). 

In the next lemma, we define the radialisation mapping Radial : 

Lemma 4.1. For a function h G C°°{N''g)^ and (x, y) G Mo, the following: 

Radial y) — h{k~^x, t) , where y — k{0, 0, t) for some k E K. 

defines a K -invariant function Radial (/i), that is smooth on No- 
Radial is an isomorphism between the topological vector spaces 
whose inverse is Rest . 

Proof. For a function h G C°°{No)^ and G No-, it is easy to see that Radial y) 

is well defined and invariant. 
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Let US show that Radial (h) G C°°{N'o)^ ■ We choose a basis {Aj).^^ ^ 3 for the Lie algebra 
of K. At a point (xo,?/o) ^ M) (with ?/o = ^o(^0)0,0), to = bol 7^ 0) we choose a local 
coordinate system (x, = (x, 0, 0)) = a(x,k,t), where x G M^, t G and k varies in 
a small two-dimensional surface in K containing ko and transversal to koK'. This change of 
variables does not affect the derivatives in x, whereas 

dy, = Cjfi{k,t)dt+ Cjj'{k,t)Aj,. 

i'=l,2,3 

By homogeneity, Cj^ G C°°{Kk x MJ') is homogeneous of degree in t, and the Cjji G 
C°°{Kk X M^) homogeneous of degree (—1) in t. More generally, we can write the derivative 



9i = dl\dlldll , J=(^l,^2,^3)GN^ 



as: 



(2) dl = J2ciAk,t)drAfAfAi\ 

where the sum is over J = {jo, ji, j2, js) G N^, with \J\ = |/|, and the cjj G C°°{Kk x MJ') 
are homogeneous of degree (jo — in t. As the function (x; /c,t) 1-^ h{k^^x,t) is smooth 
on X i^' X M, ([2]) implies that Radial /i is smooth on A^o. Furthermore h G C^i^M'^) ^ 
Radial /i G C°°{Afo) is continuous. □ 



Lemma 



4.1 



implies that the mapping Rest is a 1-1 on C°°{Af)^ and on S^JV)^ . Let us 
determine Rest {C°°{N)^). We will need the following notation: 

• For / G C°°(A/')^, we denote by PlPix^t) the homo geneous polynomial of degree M 
in the Taylor expansion of /(x, ■) at y = 0: 



For g G C°°{N'')^ , we denote by Q^^j{x,t) the homogeneous polynomial of degree 
M in the Taylor expansion of g{x, ■) at t = 0: 

Q^^\x,t) = ^d^^g{x,0)t^. 
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We see: 

g^est /) _ j^^g^ pU) ^j^^g ^^^.^^ g(Rest /) ^ ^a) _ 

Thus a function G C°°{N'')^ that is the restriction of some function / G C°°(A/')^, 
necessarily has the following property: 

Property (R). For any M G N, Radial (Q^^j) extends to a smooth function on M which 
is a homogeneous polynomial in y of degree M , with smooth coefficients in x. 

It turns out that this condition is also sufficient: 

Proposition 4.2. Let g G C°°{U')^' . 

The function g is in the image of Rest if and only if it satisfies Property (R). 

In this case, Radial {g) extends to a K -invariant smooth function f onM , whose restriction 
to Af' is g and we have Q^l) = Rest[P|{^]. Moreover if in addition g G S{J\f')^ , then 
Radial {g) G S{J\ff 

In the proof, we adapt the ideas of the Euclidean setting |Hel99[ Theorem 2.4]. 

Proof. Let g G C°°{N'')^ satisfying Property (R). For each M, we denote Pm the extension 
of Radial {Q^l)) to a smooth function on M that is a homogeneous polynomial in y of degree 
M, with smooth coefficients in x. 

Let Mo G N. The Taylor Formula gives: 

(3) ^(a;,t)-^Qf (a;,t) = — / {l-wf'"{dr"+'g){x,wt)dw. 

Let Jo G with = + 1. We have on AT^: 



Mo 

Radial (g) = 9^° ( Radial (g) -^Pj] = d^" 

j=0 



/ Mo \ 

Radial [g-Yl 



Now for any {x,y) G Mo, y 7^ can be written y = k(0, 0,t), t G M*, k E K, and by ^ and 
(j3|, ((9^° Radial ((7)) {x, y) can be written as the sum over J G N"^, \ J\ = Mo + 1, of: 

'-^^^^^ I' (1 - wf^d^A^A^Af {k~'x,wt)] dw. 

This last term remains bounded if < \y\ = |t| < 1 because cj^^j is homogeneous of degree 
jo — {Mo + 1). This implies that dy°Ra.dial (g) is bounded on a compact neighborhood of 
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(x,0) for any x, and any Jq and Mq. It is easy to see that for any / G N'^, (9^(9^° Radial ((?) 
satisfies the same conditions. Local boundedness of all derivatives is sufficient to imply that 
Radial {g) has a smooth extension to A/". □ 



We deduce easily from ([]J) the following characterisation of 7l{S{Af) 

Proposition 4.3. Let G e 5(iV') . The function G is in TZ{S{N) ) if and only if either 
of the following equivalent conditions is satisfied: 

(i) J^tG (identified with a function on M' ) satisfies Property (R); 

(ii) denoting by J^x,t the Fourier transform with respect to the variables x eM? and t G M 
given by: 

Tx,tG{x,i)= [ G{x,t)e-'''-^e~''-'dxdt , 
^x,tG (identified with a function on Af' ) satisfies Property (R); 

From G. Schwarz's Theorem, it follows (compare with |Hel99[ Theorem 2.4]): 

Corollary 4.4. Let G G 5(A^') . The function G is in TZ{S{N) ) if and only if either of 
the following equivalent conditions is satisfied: 

(i) for each j G N there exist Schwartz functions aj^i G 5(]R), i = 0, . . . , j satisfying: 



VxgM^ f G{x,t)tUt = y^aij{\ 
Jr 



2W* . 

3 ' 



x\ )x 



(ii) for each j G N there exist Schwartz functions hj.i G 5(]R), z = 0, . . . , j satisfying: 

VCgM^ / f G{x,t)t^e-'''-^dtdx = y^hij{\C\^)Q. 
Jr^ Jr • n 



i=0 

5. Proof of Theorem 13.51 



Here we give the proof of Theorem |3.5[ It is based on the properties of mappings explained 
in Section [4] and on results already shown on the Heisenberg group |ADR07[ IADR08] . These 



two key ingredients are used in the proofs of a "Geller-type" Lemma (Subsection 5.2) and 



of Theorem 3.5 (Subsection 5.3) 
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5.1. The Gelfand pair {K' x N',K'). We easily clieck tliat 

p'{x,t) = (|xp,t^X3t,a;3), 
defines a Hilbert mapping of {M'^K'), wliicli satisfies: 

p'{x,t) = [p\j^>{x,{0,0,t)),xl) and V' = Vp>. 
From tlie Heisenberg case |ADR07t IADR08] . we deduce: 

Lemma 5.1. For any G G S{N')^ , there exists g G 5(]R'^) with Q'G = g\s^,- 
Furthermore the mapping X)/) is continuous. 

Precisely, continuity of tlie last mapping means that 

Va G N 3G = G{a) > 3a' G N VG G S{N'f' 
(4) 3~geS{R') ~g\^,,=g'G II^IL,^^ < C II^L, ^, • 

Notice that the extension g depends on the Schwartz semi-norm ||.||(j]k4. 

5.2. The Geller-type Lemma. In this subsection, we will state and prove a "Geller-type 
Lemma", extending |Gel80l RDR08] . For this purpose we will need the following remark. 

Let F G S{N)^ . The mapping 

gF{R^, = j F{x,y)e-'^'''dxdy, 

is a Schwartz even function on M; by Whitney's Theorem, there exists a Schwartz function 
/„ G S{R) such that 

Vi?GM = ^F(i?^0,0); 



by Hulanicki's Schwartz Kernel Theorem or Proposition 3.3[ fo{L) is a convolution operator 
with a i^'-invariant Schwartz kernel which we denote by QF{L, 0,0) (for brevity reasons, in 
this section we will often denote a convolution operator and its kernel by the same symbol). 

Proposition 5.2 (Geller-type Lemma). Let F G S{N)^ . There exist Fi G S{N)^ and 
F2 G S{N)^ satisfying: 

F = g{F){L,0,0) + AFi + DF2. 



16 
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Proof. Let F be in S{N)^ and G = TZF G S{N')^ . By Lemma 5.1 there exists g E S 



K' 



with = Q'G. By Proposition 3.3 the operator given by: 



{L, wA, 0, 0) dw, 



w=Q 



is a convolution operator with a i^-invariant Schwartz kernel which we denote by Fi G 



S{N)^ . By spectral calculus, we have 



AFi=^(L,A,0,0)-^(L,0,0,0). 



We will have finished the proof of Proposition |5.2| once we have shown: 

(5) 3F2 G S{Nf F-gF (L, 0, 0) - AFi = DF2 

We denote by if G S{N)^ and / G S{N')^ the functions given by: 

H = F-gF{L,0,0)-AFi = F-g{L,A,0,0), 
I = TZH = G -g{L\A',0,0). 

The Gelfand transform of / is given by: 

(6) g'l M = g'G M - ~g (L>'(0), A>'(0), 0, 0) . 
On the singular part of the spectrum, ^ yields to: 

g'l (/i^, ^) = ^ (C' + r\ 0, 0, r^) - ^ (C' + r\ 0, 0, O) = 0, 
because g {C + r^, 0, 0, r^) = ^ (C^ + r^, 0, 0, 0) as g'G = o H; this implies: 

(7) VxgM^ / l{x,t)dt = 0. 
On the regular part of the spectrum, ^ yields to: 

g'l (/x^.^^ J = ~g (A(2/ + 1) + r^ A^ Ar, r') - ~g (A(2/ + 1) + r^ A^ 0, O) 
and in particular for r = 0: 

^7 (/x^.^^ J = ^ (A(2/ + 1), A^, 0, 0) - ^ (A(2/ + 1), A^, 0, 0) = 0; 
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this implies for all A > 0: 

V/gN j /(x,t)e-*^*A j c/xrft = 0, 

and {i2;}i6N being an orthogonal basis of L^(]R+), 



Eventually, we get: 



Let us set: 



Wx eM^ / /(x, xs; t)e~'^^dx^dt = 0. 



Vx G , Vt G M / I{x, x-s; t)dx3 = 0. 



rt 

G2{x,t) = I I I {x,w] s) ds dw. 



OO J —CO 



Because of ^ and (|8), we see that G2 G S{N')^ . Let us show that J-'x,tG2 (identified with 
a function on A/"') satisfies Property (R). We have for t 7^ and X3 7^ 0: 

J'.,tG2{x, i) = {X3iy'j^.,tl{x; i) and g£'f ^H^; ^1 = {UT^Qfr''\x- 1). 



By Proposition 4.3 ii), as / = TZH , J^x,tl (identified with a function on M') satisfies Prop- 
erty (R), that is Radial ( Qtv^'''*^'' ) extends to a smooth K-invariant function on M which is a 



homogeneous polynomial in y of degree M, with Schwartz coefficients in x. By G. Schwarz's 
Theorem, there exists a function Qm G C°°(]R^) of the form: 



satisfying: 



That is: 



Radial (qfr"'^) = Qm o p. 



Q^^^''\x,t) = QMi\x\^P,Xsi) = J2 c,{\x\')i'''{xsiy 

2ji+j2=M 

Because of (Isl), we have: 



Vx G , Vt G M J^x,tl{x, 0; t) = 0, 

thus the terme Cj(|xp) with j = (ji,0) is zero: we can factor out one (xsi). This implies 
that for M > 0, Radial (Q]^^!^'^ ^ (x;t)) extends to a smooth function on Af which is a 
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homogeneous polynomial in y of degree (M — 1), with smooth coefficients in x. Thus Tx,tG2 
satisfies Property (R). By Proposition 4.3 ii), there exists F2 G S{N)^ such that 7IF2 = G2- 



As D'G2 = I = TZH and TZ being 1-1 on i^-invariant functions, we obtain DF2 = H. This 
proves □ 



Applying recursively Proposition 5.2, we obtain F as a sum of functions of the form 
[Q [A^'^D^^Fj]) (L,0,0) with a rest. As the degrees of homogeneity of the operators D and 
A with respect to the variable y are three and four respectively, we will be interested in a 
sum over 2ji + j2 < M: 

Corollary 5.3. Let F G S{N)^ . There exists a family {Fj)j^j^2 of Schwartz functions 
Fj G S{N)^ satisfying for any M G N; 

F- [g [ A-^'i Fj] ) (L , , 0) = ^ A^'i D^^ Fj . 

2ii+J2<M 2ji+j2=Af+l 



5.3. End of the proof. Here we complete the proof of Theorem 3.5 Let F G S{N)^ , 



G = TZF, f = QF, g = Q'G, Fj, the associated functions by Corollary 5.3 (consequence of 
the Geller-type Lemma), and fj = GFj, j G N^. By Lemma 5.1 we choose g, gj G iS(]R^) 



Schwartz extensions of g and Q'iJlFj), j G N^, respectively. We set gpi = g o p'. 

Let us fix M. For ^ = (^,^3) G M^ setting r = ^3 and A; = |f|V(2/ + 1), we have 
p'(^. A/) G Sx)' and: 

~9A^Ai) = gop'{^,Xi)=g{Xi{2l + l)+r\XlXir,r^) 
= fiXii2l + l)+r^XlXir) 
= E A^^'^(A,rrV,(AK2/ + l) + r2,0,0) 

2ji+i2<M 

+ J2 [XirY'^fAM'^l + 1) + r^ A?, Xiv). 

2ii+j2=M+l 

Thus: 

l^p'(e,AO- Af■^(A,ry7,(A,(2/ + l)+r^0,0)| 

2ji+j2<M 
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< 



y2ji+j2=M+l 



iM+l 



This characterises the Taylor expansion of gp'{^, .): for ^ = {^,^3) first with ^ 7^ 0, and then 
for all ^, we have: 

2ji+j2=M 

= E (P2(e,t)r(P3(e,t)r/.(p'i(e,t),o,o). 

2ji+j2=M 



This shows that gp/ satisfies Property (R). By Proposition 4.2, there exists /i G S{Af)^ 
such that Rest /i = Qp/ and /i = Radial (^p/. By G. Schwarz's Theorem (see also |ADR08[ 
Theorem 6.1]), there exists / G iS(]R'^) such that /i = / o p- We have: 

Rest f = gp, = f o p\j^, = go p. 

For any point s = (A(2/ + 1) + r^, A^, Ar) G Sp, the point s' = (s,r^) G Sp/ is in imp'; it 
follows that s is in imp and f{s) = g{s') = g{s') = f{s). Thus / is an extension of /. 

Now that we have shown that the Gelfand transform of a function F G S{N)^ admits 
a Schwartz extension, we still have to prove the continuity of F G 5(iV)'' ^ QF e 5(Sp). 
We will use the following two lemmas. The first one states the improvement due to Mather 
[Mat77j of G. Schwarz's Theorem as well as some straightforward consequences: 

Lemma 5.4. Let (pi, . . . ,pg) he a minimal and homogeneous Hilbert basis for {MP,K), and 
p the corresponding Hilbert mapping. 

The induced application p* : h ^ h o p on S{E3) is split- surjective, i.e. it admits a linear 
continuous right inverse a : S{W'y^ SiW^) for p* , that is p* o a is the identity mapping 
ofS{W)^. 

We fix such a. For any h G 5(im p), the function ho p is well defined and in SiMP)^ , the 
function h = a{h o p) G 5(]R'') defines a Schwartz extension which we will call the Mather 
extension of h E iS(imp). We have: h o p = h o p. The linear mapping h 1 — > h of S{M.'^) is 
continuous. 
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It is easy to check that the Hilbert mapping, p, of (A/", K) is minimal and homogeneous. 
The second lemma follows from Rest being a 1-1 continuous mapping, from the Closed 



Graph Theorem and Lemma 5.4 



Lemma 5.5. To any g G 5(]R^) such that there exists f G iS(]R'^) satisfying f o p|_^/ = g o p' , 
we associate the Mather extension fi of f . The mapping g ^ fi is well-defined, continuous 
and linear: 



(9) 



Va G N 3C > 3a' G N 



/i 



<C\\~gh. 



Let ao G N. Let ai corresponding to a' in (g for a = an- 

Let F G S{N)^, G = TZF, f = QF, g = g'G. By Q, there exists 02 G N such that we 
have indepently of G: 

As 71 is continuous, there exists 03 G N such that we have indepently of F: 



Thus we have: 



/i 



<C,\\~go\\auR^<C2\\Gt,^^^,<Gs\\F\ 



a3,N ■ 



Notice that 03 and C3 depend only on Oq, and that /i depends on F and also on because 
cjo depends on ai. 



Appendix A. 



We adopt again the notation of Section [3] and assume that {K x A^, K) is a Gelfand pair. 
Here we give the proofs of Propositions |3.1[ |3.2 and 3^ 



A.l. Proof of Proposition 3.1 This proof is an easy generalisation of |ADR08[ Lemma 
5.3] which is a similar result given in the case of the Heisenberg group, using |B JR90j . 

Let us check that the operators of D(A^) are symmetric. In fact, G^{N) is equipped 
with the Hilbert inner product (^1,^2) = Fi{n)F2{n)dn. For any D = Dp G D(A^), 
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P e V{Af), we have {DFi,F2) = {Fi,DF2) because: 



nSO{3) 
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(10) 



P{t-^du) [ Fi{nexp{y2ujEj))F2{n)d 
Jn 

~ I " 

r ^ 



\u=0 



after the change of variable rii = nexplY^j^-^ 



Let us recall some facts about Gelfand pairs of the form {K \x N, K) |BJR90j . Let be the 
set of (the classes of) unitary representations on A^. For each n & N, let K.,^ be the stabilizer 
of TT in K. There exists a decomposition of the Hilbert space Ti.Tr into finite-dimensional 
irreducible subspaces T-Cn,a under the projective action of K^, on Hn- Each bounded K- 
spherical function on is in 1-1 correspondence with n and a, in the sense that = 0^^^ 
can be written as: 

(j){n) = I {TT(kn)u,u)dk, 



K 



where u is any unit vector in 7i7r,a (and dk the Haar probability measure of K). Let D G 
D(A^)'^. For each vr G iV, each subspace Tijr.a is an eigenspace for the operator dTx{D) and 
its eigenvalue is /Xjr.a.D satisfies: Dcl)^,^^ = f^n,a,D4>n,a- 

Note that the trace tr-^^ of operators on T-Ct^ can be computed as the sum over a of traces 
trT-^^^ of operators on 7i7r,«- 

We denote by (3 the Plancherel measure on A^: 



tr^^ [7r(F)7r(F)V/5(vr) 



F G C~(Ar). 



N 



Now let us prove Proposition |3.1 

Let D G B{Nr. It is easy to see that there exists a unique self-adjoint extension of D, 
whose domain is the space of function F G L'^{N) satisfying: 



/ V \f^.,a,D\hTn^ . [7r(F)7r(F)*] dp{n) < oo. 



Let us also denote by D the self-adjoint extension. Following |ADR08t Lemma 5.3], we 
construct a realisation E = E^ of the spectral resolution of D in the following way. Given 
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uj a Borel subset of M, we define the operator E{uj) on L'^{N) by: 

■k{E{uj)F) = ^Xa;(/^7r,a,L>)7r(F)n^,„, 

a 

where is the characteristic function of u and 11^,0 the orthogonal projection of Tin onto 
Hn^a- Then E = {E{uj)} defines a resolution of the identity, and for F G S{N), 



[ ^dEiOF = DF 



Therefore E = Ed is the spectral resolution of D. 

One readily checks that if Di,D2 G D(A^)''^, then for any Borel sets ui, uji-, the operators 
Edx{^\) and E£)^{u2) commute. 



A. 2. Proof of Proposition 3.2[ Let us recall the definition of the joint spectrum of a given 



strongly commuting family of self-adjoint operators Ti, . . . , (densily defined) on a Hilbert 
space H: it is the set 5't'i,...,t, of the g-tuples fi = (/ii, . . . G for which there do not 
exist bounded operators Ui, . . . ,Ug on H satisfying: 

j=i i=i 

Let (pi, . . . , Pg) be an ordered Hilbert basis for {Af,K) and Vp the associated family of 
strongly commuting self-adjoint operators on L?'{N). 

For each vr G A^, we decompose its Hilbert space Ti-,, = ©qTYtt.o as in the proof of Propo- 



sition |3.1| in Section |A.1| and we have for j = 1, . . . ,q: 

This implies the inclusion Sx)^ C Svp- 

For the converse inclusion, we will need the following Lemma, an easy consequence of the 
Plancherel formula: 

Lemma A.l. // a function m is continuous and compactly-supported on the Gelfand spec- 
trum, then there exists a K -invariant function M G L'^{N)^ whose Gelfand transform is 
m. Furthermore, the convolution operator with kernel M defined on C^{N) extends to a 
bounded operator on L'^{N) with operator norm sup \m\. 
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We will also use a dyadic decomposition on IR+: there exists a smooth, non- negative 
function ip, supported in the interval [^,2] and satisfying: 



Vx>0 XI ^(2' 



We set ilJa{.x) = ip(2 "x) if a > 1, and ipo{.x) = J2a<i'^i'^ "^)- 

Let /i° = . . . , fi°) G ]R'^\Sx)p. We define the fonctions maj, a > 0, j = 1, . . . ,q by: 

/i° — fij 

Each function j is continuous and compactly supported in Ex)p and because /i° is not in 
the closed set Sx)p, there exists a constant C = C{fi°) > 0, independent of a and j, such 
that: 

sup \ma,j{n)\ < (72"". 

We denote by Uaj the convolution operator whose kernel admits Gelfand transform; 

by Lemma A.l , this operator is bounded on L'^{N) with norm less than C2^"'. The operator 



J2a>o ^a,j is thus also a bounded operator on L^{N), which we denote by Uj. We check that 
for any representation vr G A^, we have on each subspace T^jr.a: 



7r(f/, 



from which we deduce: 

i=i 3=1 



This implies: 



i=i i=i 



that is, fi° is not in the joint spectrum Sd . 



This shows the inclusion T,x>p D Sxip and concludes the proof of Proposition 3.2 
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A. 3. Proof of Proposition 3.3. With the Plancherel formula (see proof of Proposition 3.1 
in Subsection A.l), it is easy to see that if m G 5(]R'^) and if m{T>p) is a convolution operator 



whose kernel is M G S{N)^ , then the Gelfand transform of M coincides with m on Sxjp. 



The proof of the rest of Proposition 3.3 relies mainly on the generalisation |ADR08t The- 
orem 5.2] of Hulanicki's Schwartz Kernel Theorem. 

We will also use the following Lemma which is well-known to specialists, but for which we 
were not able to find a written proof: 

Lemma A. 2. Let N be a graded Lie group, A/" = Vi © V2 © . . . its graded Lie algebra, (Xj) 
a basis ofVi, L = — ^Xf the associated sub-Laplacian. 

For any homogeneous left-invariant differential operator D on N of degree 2d, there exists 
a constant C = C{D) > such that we have: 



I2 ■ 



(11) \/FeC;^{N) \\DF\\^<C\\L'^F\ 

Furthermore D = 2CL'^ — D is a positive Rockland operator on N. 

Proof. We refer to |FS82t ch.G.A] for the definition and the properties of kernels of type 
a G [0,Q[, where Q is the homogeneous dimension of the group. We will also use the fact 
that the sub-Laplacian, L, has a fundamental solution, - L being a homogeneous positive 
Rockland operator of order two - and that the same is true for L'^, d = 1,2, . . .. Let us 
denote Gd a fundamental solution of L'^. For 2d < Q, Gd & C°° {N\{0}) is homogeneous of 
degree 2d - Q jFol75] . 

For any composition of left-invariant vector fields = Xi-^^Xi^ ■ ■ -Xj^ with k < 2d, it is 
easy to check that X^Gd G C°°(X\{0}) is a homogeneous function H of degree —Q + 1, 
smooth away from the origin. One further differentiation gives a homogeneous distribution 
of degree —Q. Being a derivative, it automatically satisfies the cancellation condition (63) 
of |Ste93t ch.XIII.5.3]. In fact, let be a function supported on the unit ball and normalized 
in the C^-norm. For any X G Vi and r > 0, 

(X/f,0(r-)) = -r / H{x)X(P{rx)dx = - [ H{x)X(P{x) dx, 
Jn Jn 

which is bounded independently of (p and r. 
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This implies that for every / of length 2d, the kernel X^Gd satisfies the L^-boundedness 
condition (6.3) of |FS82t ch.6.A], and thus is of type 0. The operator X^L~'^ being L'^{N)- 
bounded, we have: 

(12) ^FeC^{N) \\X^F\\^< C\\L'^F\\^. 

U 2d > Q, L'^ does not have a homogeneous fundamental solution, but, according to 
|Gel83] , it has a fundamental solution Gd which is the sum of two terms, one homogeneous of 
degree 2d — Q, and the other of the form P{x) log where P is a polynomial, homogeneous 
of degree 2d — Q, and is any smooth homogeneous norm on A^. This implies that, if the 
length k of I satisfies 2d ~ Q < k < 2d, then X^Gd is a homogeneous function of degree 



—Q + 2d — k. We can then repeat the previous argument to conclude that (12) holds for 
every d. 

Let D be a homogeneous left-invariant differential operator on of degree 2d. As D 
can be written as a linear combination of monomials X^ , with / of degree 2d, we see that 



the property (12) implies (11). Let C = C{D) be the L^-operator norm of DL^'^. In 
particular the L^(A^)-norm of the operator D{CL'^) ^ is one and / — \D{CL'^) ^ is an 
invertible operator on L'^{N). The differential operator D = 2CL'^ — D is a 2(i-homogeneous, 
left-invariant, symmetric and positive on C^{N). To finish the proof, it remains to prove 
the defining property of Rockland operators, that is, for any non-trivial, irreducible, unitary 
representation vr of A^, '/r(D) is injective on smooth vectors; this is true because we can write: 

n{D) = n{2CL'^ - D) = 2Cn (^I - ^D{CL'^y^^ 7r(L)^ 
and / — ^D{CL'^) ^ is invertible and L a Rockland operator. □ 



Before proving Proposition |3.3| let us define some notation. We equip the two-step nilpo- 
tent Lie algebra A/" with an Euclidean product such that K acts orthogonally. K stabilises 
the centre Z of A/", and its orthogonal complement V = Z^. The decomposition A/" = V © 2 
endows A^ with a structure of graded Lie group. Q = dimV + 2dim^ is the homogeneous 
dimension of the group. For the symmetrisation mapping, we assume that the basis {E. 



p 



is given as a basis (-Ej)fii of V completed with a basis {EiY- of Z. As the action of 
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K on V{M) respects the degree-graduation in both the Z and V-variables, there exist bi- 
homogeneous Hilbert basis {pi, . . . ,pg} in the sense that each polynomial pj is homogeneous 
in the ^-variables and in the V-variables. For a bi-homogeneous Hilbert basis {pi, . . . ,Pg}, 
we denote by the degree of homogenity of pj in the V-variables, and by the degree 
of homogenity of pj in the ^-variables; dj = d^ + ^d^ is the degree of homogeneity of the 
operator Dp^ for the structure of graded Lie group of A^. 



Let us start the proof of Proposition 3^ We notice that it suffices to show the result for 
one Hilbert mapping because of the existence of a polynomial mapping between two Hilbert 
mappings. We choose a bi-homogeneous ordered Hilbert basis p = (pi, . . . , p^) with the two 
following properties. First PiC^^^iUjEj) = Xl^ii I'^jP- Second, the polynomials pi, . . . , Pq^ 
are of even degree of homogeneity in the V-variables and the polynomials p^^+i, ■ ■ ■ , Pq are 
of odd degree of homogeneity in the V-variables. 

Let m be in iS(]R''). S denotes the set of all the sequences e : {gi + 1, . . . , g} — > {0, 1}. Using 
Whitney's Theorem or G. Schwarz's Theorem, there exists a family of Schwartz functions 
(^e)egs') ^ 5(]R'') Satisfying for all (ri, . . . , r^) G M'^: 



m(ri, . . . , Tg) = ^ r'm,(ri. 



2 2\ 
1 ' qii ' gi+l' ■ ■ ■ 1 ' q)i 



where we use the notation r'^ = r^^^^l^^ . . . rl^'^\ 

The operator Di = Dp^ is the sub-Laplacian of which is a positive Rockland operator. 



By Lemma A. 2 there exist constants Cj, j = 2, . . . ,q such that 

it 

• for j = 2, . . . , gi, the operator Dj = —Dp^ + CjDp^ is a positive Rockland operator 
on A^ 

• for j = gi + 1, . . . , g, the operator Dj = —D'^^ +CjDpl is a positive Rockland operator 
on A^ 

For r = (ri, . . . , r^) G M*^, we set [A(r)]-,^ = ri and: 

[A{r)]. = -Tj + Cj/p"^ , j = 2, . . . , gi 
[A{r)]. = -Tj + Cjvf' , j = gi 1, . . . , g 
This defines an application A : ^ which is a C°°-diffeomorphism of M'' and whose 
Jacobian equals (— 1)"^"^ at any point. Thus if h is in iS(]R'^) then h o A~'^ is in 5(]R'^). 



We have: 
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(13) m(D,,, . . . , D,^) = D;,MDp,, . . . , D,^^ , D^^^^, D^) 



e5 



(using the notation D^, = -Dp,j+i • • • ^Pq ) ^^"i- 
(14) me(I^p„ . . . , D,^^ , Dl^^^, DD = o A-\D^, ...,D,) 



Each operator given by (14) is a Schwartz multipher o A^^ G 5(]R'') of a strongly commu- 
tative family of positive Rockland operators Dj, j = 1, . . . q. By |ADR08l Theorem 5.2], it is 
a convolution operator with a Schwartz kernel M^^^^-i (•^^-j. Because of the expression (13), 
we deduce that the operator m{Dp^, . . . , Dp J is also a convolution operator with a Schwartz 

kernel Mm,v, = E.e5 ^p'^m,oA-i,(D,)- 

The continuity of m G iS(]R'^) t— > Mm,Vp ^ S{N)^ is a direct consequence of the following 
facts: 

• by Schwarz-Mather's Theorem, the mappings m G iS(]R'') i-^ G iS(]R''^ x [0, oo['^^''^), 
e G S", are continuous 

• the application A being a C°^-diffeomorphism of W with (— 1)'^"^ as jacobian, the 
mapping A'^* : h G S{W) ^hoA-^e S{W) is continuous 

• by |ADR08t Theorem 5.2], the application that maps m G 5(]R'') to the kernel ^ -j 
of the operator m(l)i, . . . , Dq) is continuous 



The proof of Proposition 3.3 is thus complete 
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